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ABSTRACT 

California's demonstration programs in reading and 
mathematics are the State's only educational programs whose funds are 
used exclusively to serve low-achieving students from disadvantaged 
areas at the junior high-middle school level. The purpose of the 
programs is threefold: to establish exemplary programs in reading and 
mathematics instruction in Grades 7, 8, and 9; to develop the reading 
and mathematics competence of low-achieving students; and to 
disseminate these successful practices to other educators. The 
findings of a series of annual reports submitted to the Legislature 
since 1970 show that the demonstration programs have adhered closely 
to the legislative mandate in serving those students in disadvantaged 
areas. They also show that low-achieving students in reading and 
mathematics can equal or surpass expected standards of achievement as 
the result of the programs 1 intervention. The report shows, moreover, 
that the demonstration programs have operated with increasing 
efficiency over the years, both in use of funds and in promoting 
student achievement. Finally, the demonstration programs* information 
and materials have been extensively adapted and utilized by other 
educational professionals throughout California as the result of 
dissemination activities. The State Department of Education 
recommends that the current demonstration programs in reading and 
mathematics be continued because they have been effective in 
promoting student achievement. Further, the legislature should 
consider three additional options: (1) obtaining new information on 
demonstration program practices; (2) expanding these practices to new 
schools; and (3) developing new demonstration programs in other 
curricular areas. (KH) 
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EXECUTIVE SUMMARY 



The demonstration programs in reading and mathematics were originally 

mandated by AB 938 (Chapter 1578/1969)* More recently, SB 1155 (Chapter 

1270/1983) defined a sunset date in 1985 for these programs and required a 

legislative review in 1984* This report is in response to that directive* 

The demonstration programs were intended for schools in "designated areas 
of disadvantage," which were defined as areas with low-income families, high 
mobility rates, and low student academic test scores* 

The purpose of the programs was threefold: to establish exemplary pro- 
grams in reading and mathematics instruction in grades seven, eight, and nine; 
to develop in low achieving students competence in reading and mathematics; to 
disseminate these successful practices to other professional educators* 

The findings in this report were assembled from the series of annual 
reports submitted to the Legislature since 1970* The findings show that the 
demonstration programs have adhered closely to the legislative mandate in 
serving those students in disadvantaged areas* They also show that low- 
achieving students in reading and mathematics can equal or surpass expected 
standards of achievement as the result of the programs' intervention* The 
report shows, moreover, that the demonstration programs have operated with 
increasing efficiency over the years — both in use of fuads and in promoting 
student achievement* Finally, the demonstration programs' information and 
materials have been extensively adapted and utilized by other educational pro- 
fessionals throughout California as the result of dissemination activities* 

The State Department of Education recommends that the current demonstration 
programs in reading and mathematics be continued because they have been effective 
in promoting student achievement* 

It is further recommended that the Legislature consider three additional 
options that would improve the programs while maintaining their basic purposes* 
These options are: (1) obtain new information on demonstration program 
practices; (2) expand these practices to new schools; and (3) develop new 
demonstration programs in other curricular areas* 



INTRODUCTION 

Senate Bill 1155 (Chapter 1270/1983) requires the State Department of 
Education to submit a report on the demonstration programs in reading and 
mathematics to the Legislature and to the Legislative Analyst. The purpose 
of the report is to assist the Legislature in its review of this categorical 
program as required by Education Code Section 62006(a), The demonstration 
programs will cease to be operative, or "sunset," on June 30, 1985, unless 
the Legislature enacts legislation to continue the programs pursuant to the 
recommendations of the review. (See Appendix I.) 

The demonstration programs are the only educational programs whose funds 
are used exclusively to serve low-achieving students from disadvantaged areas 
at the junior high/middle school level in California. They are designed to 
improve the academic achievement of these students in reading and mathematics. 
The programs are expected to be exemplary, are mandated to be cost-effective, 
and are required to share their successful practices with other schools. 

This report contains, first, a description of the demonstration programs, 
including a description of the student population served, how the programs are 
operated at the local level, and how the programs are administered at the state 
level. The description is supported by relevant statistical data on program 
enrollments, student achievement, and funding levels. Next is a discussion of 
related federal and state programs which may be appropriate for the Legislature 
to consider in its review of the demonstration programs. Third, an analysis of 
the programs describes the methods used for the evaluation, and program findings 
report pupil achievement, program costs, and program effectiveness. Finally, 
these findings are summarized and recommendations are made for ways to improve 
the programs while maintaining their basic purposes. 
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LEGISLATIVE HISTORY 



The demonstration programs in reading and mathematico began in 1970 after 
the Legislature passed AB 938 (Chapter 1578/1969), The programs were originally 
authorized by Education Code Section 58608 to continue until 1972* However, in 
1972, Senators Marks and Moscone introduced SB 375 (Chapter 596/1972); it became 
the authorizing legislation which permitted the programs to continue until 1975* 
In 1975, Senator Rains and Assemblyman McVittie co-introduced a bill, SB 420 
(Chapter 1127/1975), which continued the programs until September 1, 1978, and 
strengthened their dissemination requirements. AB 8 (Chapter 282/1979) con- 
tinued the programs until 1982, when the Legislative Analyst was required to 
conduct a thorough sunset review study. AB 2196 (Chapter 1354/1980) removed the 
mention of a termination date by repealing Education Code Section 58608, but 
left the AB 8 sunset provisions intact. Most recently, SB 1155 (Chapter 1270/ 
1983) changed the sunset date to 1985 and required a legislative review of the 
programs in 1984 (the year prior to the sunset date) so that school districts 
and county offices of education would not be subject to disruption in the 
planning and administration of the programs. 

It was originally the intent of the Legislature that the Department of 
Education, with the approval of the State Board of Education, establish exem- 
plary programs for intensive instruction in reading and mathematics for students 
in grades seven, eight, and nine who attended school in the most seriously 
disadvantaged areas in California. The programs were intended to serve as 
demonstration projects aimed solely at developing, within these students, 
above-average competence in these two basic skill subjects. The programs also 
were intended to disseminate their successful practices to other schools which 
would replicate those practices. 

The authorizing legislation specified that the demonstration programs 
be operated by school districts. Local program approving agencies were granted 
waivers of the Education Code, if necessary, for the development of the pro- 
grams. A unique provision in the legislation required the ongoing replacement 
of projects which were deemed least cost effective by ones of proven effective- 
ness or by new projects which held promise of increased effectiveness. The 
Superintendent of Public Instruction was required to submit annual reports to 
the Legislature on the implementation and evaluation of the demonstration 
programs. 



PURPOSE 



The demonstration programs in reading and mathematics are designed to 
improve the achievement of students living in disadvantaged areas. In addi- 
tion, demonstration programs are required to act as exemplary programs and 
to disseminate information on curriculum and other educational practices to 
schools for the purpose of improving student achievement with cost-effective 
reading and mathematics programs. According to State Board of Education regu- 
lations, "exemplary program" is defined as an innovative, creative program over 
and above a districts regular program that can serve as a model for future 
programs. "Designated areas of disadvantage" are those areas which have such 
factors as the high incidence of low-income families, unemployment, persons re- 
ceiving assistance under the federal program of aid to families with dependent 
children (AFDC), and low-achieving students in the schools. 

Further, the demonstration programs are required to be designed to create 
an atmosphere or climate for learning, to motivate the students for further 
learning, and to assure educational attainment or success in reading and 
mathematics. The specific goals of the programs are: 

1. Development of creative programs to serve as models or guides for improved 
instruction, generally, and for compensatory education, specifically 

2. Experimentation with new curricula and methodology designed to meet 
the needs of low-achieving pupils 

3. Analysis and assessment of tne abilities and needs of low-achieving 
pupils 

4. Provision of individualized instruction 

5. Employment of a variety of approaches to systematic, sequential skills 
development 
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Provision of corrective programs for pupils performing one year or more 
below grade expectancy level 



PROGRAM HISTORY AND DESCRIPTION 



In the 1969-70 school year, 16 different schools in various areas of the 
state were approved by the State Board of Education to begin a demonstration 
program in reading or mathematics (or both) for their seventh grade pupils 
during the second semester of the 1969-70 school year. The schools chosen were 
those in low-income areas with high student mobility rates and low academic 
achievement test scores and whose students would otherwise have found difficulty 
in achieving future success in high school* 

History 

Since 1969-70, a total of 47 schools have implemented demonstration pro- 
grams; these schools have operated a total of 37 reading projects and 23 
mathematics projects* Schools have implemented single projects (e.g. , reading 
or mathematics) or dual projects (reading and mathematics). There were 16 
original schools when the programs began, which operated 15 reading and 13 
mathematics projects. This year, 28 schools are operating 16 reading projects 
and 13 mathematics projects. Only 4 of the 16 original schools are still 
operating a reading or mathematics project, and only 1 of them has not had a 
project terminated. (See Appendix II.) In keeping with the legislative 
requirement to terminate the least cost effective, 31 projects have either been 
terminated or have withdrawn voluntarily since the programs 1 authorization. Ten 
programs were moved from one school to another because of such unanticipated 
events as school closure or fire. (See Appendix III.) 

Description 

The demonstration programs vary from school to school. All projects place 
students with different achievement levels together in the same classroom, but 
other program features may vary. Frequently found features include individual- 
ized instruction, use of aides in the classroom, learning centers or laborato- 
ries, and specially designed curricula, materials, and activities. Each project 
is designed by the principal, project director, teachers, aides, and other staff 
at the school and is based on the needs and characteristics of the students and 
the school. Those same school personnel also make decisions to modify and 
improve their school's program as needed. 

All students at the project grade level participate in the demonstra- 
tion program at that school. The project moves with the students, serving the 
same group of students throughout grades seven, eight, and nine in a three-year 
cycle. However, since the demonstration programs were established, many partici- 
pating schools have changed from three-year junior high schools to two-year 
schools. Accordingly, projects at these schools now operate on a two-year 
rather than a three-year cycle with the same group of students. 

In addition to their goal of improving student achievement, all continuing 
demonstration programs engage in a variety of dissemination activities designed 
to provide other schools with information and materials about their successful 
efforts to improve student achievement in a cost-effective manner. All programs 
make presentations at annual dissemination conferences coordinated by the State 
Department of Education to which the education community and other interested 
citizens are invited. Each demonstration program welcomes visitors who want to 
learn about that program on a first-hand basis. Every program responds to 
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requests for curriculum materials and information by giving reports, instruc- 
tional guides, sample materials, and other information so that interested 
teachers and administrators can use project ideas at their own schools. Many 
programs use a variety of other means to spread information about their programs, 
including newspapers, television and radio presentations, slide and tape shows, 
and workshops. 

Demonstration programs are compared with each other annually, and those 
that are deemed least effective on the basis of student achievement and cost- 
effectiveness are terminated. Funds from terminated programs are used to 
replicate cost-effective programs in other schools and in other eligible 
districts or to add new, potentially effective, programs. 

Typical Operation 

A typical reading program may take half of the school* s seventh grade 
English classes in a school into a reading center for a three-week period. The 
reading teachers and aides in the center work with students in small groups or 
individually, usually on a contract basis agreed to by the teacher and student. 
Students are pretested and assignments are given only for work which is needed. 
A variety of methods and materials are used, depending on the students* needs. 

After a three-week period, these students return to their English classes 

and the remaining half of the: classes attend the reading center. The students 

alternate with three weeks in the center and three weeks in the classroom 
throughout the year. 

Although each program is different, great emphasis is placed on a sequen- 
tial, systematic curriculum with each student working on specific assignments. 

A typical mathematics program might begin each class period with a short 
quiz and quick drill for all students. Each student then begins work on 
individual contract lessons. After a few lessons, a test is taken to determine 
if the next series of lessons should be attempted or if review work should be 
done. Students take many quizzes to make certain that they understand the 
material that they have been studying and are ready to move on to the next 
level. Each student might spend every fifth day in a mathematics lab doing 
hands-on assignments related to the classroom work. 

In recent years, computers have been used in some of the programs* Several 
demonstration programs are now offering in-service workshop sessions in the use 
of computers in teaching reading and mathematics to students and in program 
ma nagement. 

Population Served 

The enabling legislation required that students served in the demonstration 
programs were to be in schools in areas designated as disadvantaged (Education 
Code Section 58602). Such schools ware in low-income areas, typically with 
large minority populations that had high mobility rates and had students with 
low academic test scores. It was hoped that intervention by the demonstration 
programs would reverse the trend of low academic achievement. Approximately 
8,000 students per year have been served by the programs. A description of the 
average demonstration program student population is as follows: 
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Ethnicity , Sixty-one (61) percent of tne average demonstration program 
school population in 1983-84 is made up of minority students. The average 
ethnic distribution is as follows: 35 percent Hispanic students, 19 percent 
black (not of Hispanic origin) students, 5 percent Asian or Pacific Islander 
students, 1 percent American Indian or Alaskan native students, 1 percent 
Filipino students* Thirty-nine percent of the students are white (not of 
Hispanic origin). This compares with 1981-82 data on the statewide ethnic 
distribution in the public schools: 26 percent Hispanic students, 10 percent 
black (not of Hispanic origin), 6 percent Asian or Pacific Islander students, 1 
percent American Indian or Alaskan native students, 2 percent Filipino students, 
and 56 percent white (not of Hispanic origin) students. A few of the schools 
have large populations of only one minority group (for example, one school has 
95 percent Hispanic and another has 93 percent black), while the majority have 
mixed student populations. 

Income level . The income level of the programs 1 designated areas is 
difficult to assess directly; however, an indirect measure is the number of free 
lunches served to the students in these areas that qualify under federal AFDC 
criteria. The average demonstration program school serves a free lunch to 53 
percent of its students. The range of free lunches served is from 23 percent 
to 93 percent. 

Academic level . The -average achievement of students entering the program 
in September 1982 was more than one year below grade level in reading or 
mathematics achievement. (An analysis of students 1 academic level is found in 
Table 4, page 13.) 

Mobility . Although mobility rate statistics have not been collected, most 
project directors have stated that the demonstration program schools have had 
much higher rates than other similar schools in their districts. 

State Administration 

The Department of Education has one consultant-manager and one half-time 
secretary assigned to administer the demonstration programs. The manager 
reviews, processes, and prepares State Board of Education agendas for demonstra- 
tion programs' applications, revisions, and amendments; offers programs assis- 
tance during regular site visits to the programs; coordinates with and provides 
demonstration program in-service workshop sessions for other Department and 
professional organization sponsored conferences; plans and conducts in-service 
programs for program staff; plans, coordinates, and conducts statewide demon- 
stration program in-service conferences; disseminates printed material about the 
demonstration programs; and serves as liaison for the demonstration programs at 
the state level. Program evaluation r distance is provided by the Division of 
Planning, Evaluation and Research, 

Funding Level and Enrollment 

To be eligible for the funding of a demonstration program, a school 
district must maintain schools serving low-achieving seventh, eighth, or ninth 
grade students in the most concentrated areas of poverty as designated by the 
Superintendent of Public Instruction under provisions of Education Code Section 
54483. 
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The programs were originally funded at the $3,000,000 level from the 
state General Fund. Because of cost-of-living increases in 1974, 1977, 1980, 
and 1983, the amount available for fiscal year 1983-84 is $3,771,000. 

The program schools are required to use regular district funds for their 
basic educational programs and to use demonstration program funds only for read- 
ing or mathematics program activities not normally provided by the district. 
Most of the program funds are used for personnel; about 85 percent is spent for 
certificated and classified salaries and employee benefits; the remainder is 
used for books, supplies, equipment, rents, leases, utilities, travel, house- 
keeping, and other services. There is no requirement for districts to match 
local monies with the extra state funds. 

Related Federal and State Programs 

There are no other federal or state funded programs which serve only 
junior high/middle school students in reading and mathematics. While the 
demonstration programs 1 funds can be spent to fund only one grade level at a 
time, districts often use other funding sources to offer a similar program to 
students in the other grade level(s) not being served; often federal or state 
compensatory education funds are used in these cases. 

The federal National Diffusion Network (NDN) funds the national dissemina- 
tion of information about validated exemplary programs in all subject areas. 
However, NDN funds only the national dissemination of programs; it does not 
fund their ongoing operations. Five of the demonstration programs have been 
validated as exemplary by the National Joint Dissemination Review Panel, and 
three have been funded for national dissemination by the National Diffusion 
Network. 

A related program at the state level is the Miller-Unruh Reading Program 
which provides funds to pay the salaries of reading specialists. However, 
this program serves elementary school students with reading disabilities; it 
does not serve junior high/middle school students. 
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PROGRAM ANALYSIS 



Introduction 



The program analysis section contains summaries of the general evaluation 
methodology used by the demonstration programs as well as summaries of the 
achievement data, program costs data, program cost-effectiveness data, and 
program dissemination data, which have been described in annual reports since 
1971. The aim of this section is to examine trends in these data and to make 
inferences about the educational significance of the program outcomes. 



General Evaluation Methodology 

Evaluation Plan 

From the outset, projects have reported evaluation data annually to the 
State Department of Education. These reports contained data about the number 
of students in a program, pre- and post-test CTBS achievement test data and 
fiscal information. This information was used primarily to establish rankings 
of effectiveness among the programs. Prior to 1975-76, rank-ordered test: score 
data appeared in annual reports. In the analysis presented herein, emphasis 
is placed on the data in the eight annual reports from 1975-76 to the present 
which contain scaled scores data as well as rank-ordered data. 

Data Components 

The data components for the demonstration programs' annual evaluation 
include pupil achievement, program costs, cost-effective indices, and program 
dissemination information. Brief descriptions of the methods used to analyze 
the data are described below. 

- Pupil achievement 

Demonstration programs are required to administer standardized 
achievement tests to students in the designated grades on a fall pretest 
and later on a spring post-test. The achievement test, typically adminis- 
tered in October and May each year, has been the Comprehensive Test of 
Basic Skills ( CTBS ), form Q, R, S, or T, level 2, 3, or 4. Achievement 
gains were expressed as increases in scaled scores from the October pretest 
to the May post-test, as measured by the subtraction of each projects mean 
pretest scaled score from the mean post-test scaled score. (The mean 
scaled scores were obtained by converting each student's raw score to a 
scaled score, summing the scaled scores, and obtaining the mean.) 

Scaled score increases from pretest to post-test were compared to 
gains predicted on the basis of the pretest percentile rank and were ex- 
pressed as a percent increase in scaled score over the increase predicted 
from October to May. For each project, the percentile rank of the mean 
pretest scaled score was read from the publishers 1 norms for October of 
the correct grade. The predicted post-test scaled score was the scaled 
score of the same percentile rank on the publishers 1 norms for May of 
the correct grade. The rationale for this calculation of predicted 
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score was that, with no program intervention, project participants would 
progress at the same relative rate as that which had determined their 
pretest percentile rank, and their post-test percentile rank would thus 
be the same as the pretest percentile. The difference between the pretest 
and the predicted post-test was calculated to give the predicted scaled 
score increase (for example, actual pretest 381, predicted post-test 400, 
for a predicted gain of 19 scaled score points). The actual difference 
between t\e obtained pretest and post-test was calculated (for example, 
actual pretest 381, actual post-tist 438, for an actual increase of 57 
scaled score points). The difference between the actual gain and the 
predicted gain was calculated ^continuing the example, 57 minus 19 equals a 
difference of 38 scaled score points). The ratio of the difference between 
the actual score and the predicted score to the predicted score (38/19) was 
expressed as the percent of increase in scaled score over the predicted 
increase (200 percent). 

Achievement gains also were expressed as months gain in mean grade 
equivalent scores for each month of instruction between pretest and post- 
test. First, the gain in grade equivalent scores from pretest to post-test 
was calculated by the conversion of mean pretest and post-test scores to 
the equivalent grade equivalent scores according to publishers 1 norms (for 
example, a pretest scaled score of 28 equals grade 4.5; a post- test scaled 
score of 39 equals grade 5.8) and subtracting the pretest score from the 
post-test score (for example, the gain from 4.5 to 5.8 is 1.3 years or 13 
months, with the school year being calculated as ten months in duration). 

The increase in scores was then divided by the actual number of 
months 1 instruction between pretest and post-test (in this case, 13 months 
divided by seven months 1 instruction, or 1.85 months 1 gain for each month 
of instruction). 

In the analyses reported, the computed gains may be overestimates of 
the actual gain. This phenomenon occurs when test publishers 1 norms are 
based on linear interpolation and intrapolation from few empirical data 
points.* 

Program costs 

With the total project expenditures used as the basis, the percentage 
of expenditures in each of six major budget categories was calculated 
across projects. Unaudited end-of-year financial reports provided the 
data. Expenditures per student were calculated by obtaining the operating 
expenses reported as of May each year and dividing by the number of stu- 
dents served by the projects. The percent of expenditures in excess of 
estimated district expenditures per unit of a.d.a. was calculated for each 
project. 



*J. L. Housden and D. R. Sweet, "Problems in Estimating Fall and Spring Norms 
for Standardized Achievement Tests.*' Sacramento: California State Department 
of Education, Office of Program Evaluation and Research, 1977 (staff working 
paper) . 
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Cost-ef fectiveness measures 



The enabling legislation specified that the demonstration programs be 
evaluated and compared with each other in cost-effectiveness. A cost- 
effectiveness formula was derived from a comparison of two variables- 
increases in student achievement compared to predicted increases and 
increases in expenditure over the ordinary or district expenditures per 
unit of average daily attendance (a.d.a.). The cost-effectiveness formula 
gave the percent of extra student achievement gained for each 1 percent 
increase in cost. For example, a cost-effectiveness index of one meant 
that there was a 1 percent increase iv achievement for each 1 percent 
increase in cost. 

Dissemination and replication activities 

The demonstration programs are required to disseminate information 
(Education Code Section 58601) about their successful practices. Informa- 
tion about this activity was obtained from evaluation reports submitted in 
June for each program. Fiscal information about dissemination and replica- 
tion activities was obtained from preliminary fiscal reports which were 
also submitted annually for each program by June. 



Program Findings 

Table 1 shows the pre/post-test changes which have occurred across eight 
years of program operation. The gains of 50 scaled score points on the average 
in reading and 64 scaled score points in mathematics indicate that the educa- 
tional practices interposed between the pretest and the post-test had a signifi- 
cant and reliable effect on the reading and mathematics levels of the students. 



TABLE 1 



Actual Cains in Comprehensive Test of Basic Skills (CTBS) 
Reading and Mathematics Scaled Scores 





Reading 


Mathematics 


Year 


Pre- 


Post- 




Pre- 


Post- 




test 


test 


Gain 


test 


test 


Gain 


1982-83 


481 


540 


59 


462 


539 


77 


1981-82 


507 


563 


56 


486 


558 


72 


1980-81 


471 


524 


53 


459 


527 


68 


1979-80 


504 


548 


44 


487 


546 


59 


1978-79 


444 


490 


46 


43S 


512 


73 


1977-78 


486 


527 


41 


463 


527 


64 


1976-77 


452 


501 


49 


448 


499 


51 


1975-76 


467 


512 


45 


455 


506 


51 


Averages 


477* 


527** 


50 


462* 


526** 


64 



*Chi square test for homogeneity p > .05 
**T test of overall differences in pretest and post-test 
p < .05 



mean 
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Table 1 shows that pupils in the demonstration programs have continually 
shown increases in reading and mathematics skills over the years. Moreover, 
these programs have become more efficient in teaching these skills, as indicated 
by the steadily increasing gain scores. This latter fact m~y be due to elimin- 
ating those programs which are the least effective in improving achievement 
levels or to increasing success in applying innovative reading and mathematics 
instructional principles. 

Table 2 indicates that the gains in reading and mathematics are clearly 
greater than one would have expected. Had there been no program intervention, 
students would have been expected to progress at the same rate, as indicated by 
the pretest. In fact, their post-test rate was accelerated by 203 percent over 
the predicted gain on the average in reading and by 392 percent in mathematics. 



TABLE 2 



Percent of Increase of Actual Gains vs. Predicted Gains 
in CTBS Reading and Mathematics Scaled Scores 





Reading 


Mathematics 






Pre- 


Percent 




Pre- 


Percent 




Actual 


dicted 


in- 


Actual 


dicted 


in- 


Year 


gains 


gains 


crease 


gains 


gains 


crease 


1982-83 


59 


17 


247 


77 


12 


542 


1981-82 


56 


18 


211 


72 


14 


414 


1980-81 


53 


16 


231 


68 


12 


467 


1979-80 


44 


17 


159 


59 


13 


354 


1978-79 


46 


16 


188 


73 


12 


508 


1977-78 


41 


17 


141 


64 


14 


357 


1976-77 


49 


15 


227 


51 


14 


264 


1975-76 


45 


15 


200 


51 


14 


264 


Averages 


50 


16.5 


203 


64 


13 


392 



As illustrated in Table 2, the reasons for this accelerated pace may be due 
to the educational conditions which prevail in the typical demonstration program 
as follows: 

Students are taught individually or in small groups for ^t least part of 
each day. 

Most students study in a learning laboratory at least once per week, 
working with highly trained teachers, tutors, and aides in learning 
activities designed especially for then. 

Students with different abilities are grouped together (called heterogeneou* 
grouping). 

Learning activities are planned and directed by the staff of each 
participating school. 
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Learning materials and activities are prescribed on the basis of a diagnosis 
of each student 1 s ability and learning style. 

Students are made aware of their successes and of the high expectations 
held for them. As a result, they are motivated to learn more effectively. 

- School programs rated least cost effective are terminated. 

Table 3 translates this accelerated growth into months of gain for months 
of instruction. A common educational standard of growth is a one-month gain for 
each month of instruction. However, in the demonstration programs, students 
gained 2.3 months for each month of instruction in reading and 2.9 months for 
each month of instruction in mathematics. 



TABLE 3 



Month-to-Month Gain in Reading and Mathematics 
Achievement, as Indicated by CTBS Testing 





Month's gain in 


Month' 8 gain in 


Year 


Reading 


Mathematics 


1982-83 


3.0 


3.4 


1981-82 


2.7 


3.0 


1M80-81 


2.2 


2.8 


;79-80 


2.4 


3.1 


1978-79 


2.C 


3.6 


1977-78 


2.0 


2.9 


1976-77 


2.0 


2.3 


1975-76 


1.9 


2.1 


Average 


2.3 


2.9 



These results suggest that the demonstration programs can be effective in 
reversing the trend of persistently low academic achievement of students from 
schools in economically disadvantaged areas. 

Table 4 shows that students entering the demonstration programs for the 
first time in the seventh grade had a CTBS grade equivalent score of 6.1 grades 
in reading and 5.6 grades in mathematics. This conforms the program intent to 
serve academically underachieving students. 
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TABLE 4 



Pretest and Post-test CTBS Grade Equivalents in 
Reading and Math Achievement for Seventh Grade Students 





Reading 


Math 




Number 


Pretest 




Number 


Pretest 






of 


equiva- 


Post-test 


of 


equiva- 


Post-test 


Year 


projects 


lent 


equivalent 


projects 


lent 


equivalent 


1982-83 


' 15 


6.7 


8.4 


10 


6.4 


9.1 


1981-82 


1 


7.6 


9.7 


3 


5.1 


7.9 


1980-81 


10 


5.9 


7.7 


8 


6.0 


8.5 


1979-80 


1 


6.4 


7.8 


1 


5.4 


7.1 


1978-79 


12 


5,7 


7.3 


7 


5.6 


8.4 


1977-78 


2 


4.9 


6.1 


2 


5.2 


7.0 


1976-77 


6 


5.5 


7.2 


4 


5.8 


7.7 


Average 




6.1 


7.7 




5.6 


8.0 



By the end of the school year, these underachieving students were at 
grade level in reading achievement and above grade level in math achievement. 

Table 5 shows operating costs per pupil in the average demonstration 
program in comparison with regular district expenditures per pupil for the 
average district conducting a demonstration program. 



TABLE 5 



District and Demonstration Program Expenditures 





•70- 
71 

I 


'71- 
72 

1 


'72- 
73 

1 


'73- '74- »75- '77- '78- '79- '80- '81- '82- 
74 75 76 78 79 80 81 82 83 
1 1 1 1 1 1 1 1 1 


•83 
84 


$2,400- 








2 » 316 2,236 




2,200- 












1,000- 
1,800- 
1,600- 
1,400- 








/ 2,011 
1, 688/ 

1,348^^7 




1,200- 
1 ,000- 
800- 


$855 


924 


1,008 


1, 188 

Average district expenditures 
(current expenses per unit of a.d.a.) 




600- 












400- 


$56T 


586^ 




JZL 370 97fi ? 92 263 281 274_j£^_297 




200- 






334 


Average demonstration program 
operating costs per student 





2,368 
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Table 5 illustrates the relative stability of the per-pupil operating costs 
of demonstration programs across several years of operation* For the same peri- 
od, regular district expenditures per unit of a.d.a. have risen substantially. 

Table 6 shows the cost-effective indexes which have been attained since 
1975—76* The cost-effective index is a measure required by the programs 1 
enabling legislation. It is based on a comparison of two variables — increases 
in student achievement compared to predicted increases in student achievement 
us * n g CTBS scaled scores and increases in expenditures over the ordinary or 
district expenditures per unit of a.d.a. The cost-effectiveness formula gives 
the percent of extra achievement gained for each 1 percent increase in cost. 
For example, a cost-effectiveness index of two means that there is a 2 percent 
increase in achievement for each 1 percent increase in cost. 



TABLE 6 



Cost-Effectiveness Index for Reading 
and Mathematics Projects 





Reading 


Mathematics 


Year 


Number of 
schools 


C/E index 


Number of 
schools 


C/E index 


1982-83 


16 


20.0 


13 


55.0 


1981-82 


14 


13.8 


12 


33.4 


1980-81 


16 


14.8 


12 


42.8 


1979-80 


17 


14.4 


12 


28.5 


1978-79 


15 


9.7 


9 


44.4 


1977-78 


14 


8.4 


9 


31.4 


1976-77 


14 


10.8 


8 


24.4 


1975-76 


16 


7.2 


8 


14.4 



Since the cost-effectiveness formula was first developed, there has been a 
steady increase in program cost effectiveness, as can be seen in Table 6. For 
the 1982-83 year, a cost-effectiveness index of 20 in reading means that the 
gain in reading achievement rose 20 percent for each 1 percent increase in 
program cost; math rose at a 55 percent gain for every 1 percent increase in 
program cost. 

Table 7 shows that, in order for the schools to implement their demonstra- 
tion programs, they used state funds to obtain personnel, materials, and 
services beyond that provided by school district resources. The percent of 
expenditures by budget categories was obtained from the unaudited project 
end-of-year financial reports. 



TABLE 7 



A Comparison of Expenditures by Selected Budget 
Categories for 1982-83, 1979-80, and 1977-78 Fiscal Years 



Fiscal 
year 


Salaries 


Employee 
benefits 


Contracted 
services* 


Materials 
and equipment 
replacement 


Capital 
outlay 


Certified 


Classified 


1982-83 


43% 


27% 


16% 


6% 


6% 


3% 


1979-30 


42% 


27% 


15% 


8% 


7% 


2% 


1977-78 


43% 


27% 


14% 


7% 


8% 


1% 



^Utilities, housekeeping, travel, rent, leases, etc. 



NOTE: Expenditures may not total 100 percent due to rounding. 



Dissemination and Replication Activities and Costs 

In addition to increasing the academic achievement levels of students, the 
goals of the programs have been accomplished by teaching other school staffs 
about the successful educational practices learned in the demonstration programs, 
as described in the following paragraphs. 

Each program publishes curriculum materials which it has developed and 
provides them free of charge to any interested school person. During 1982-83, 
the project directors estimated that they gave information about their curricula 
to 17,214 school people. 



The individual programs conduct dissemination sessions for teachers in 
their own and neighboring districts and at professional organization meetings. 
During 1982-83, the average program conducted 33 separate dissemination sessions 
to demonstrate their successful practices in the teaching of reading or mathemat- 
ics. Together, the programs conducted 736 dissemination sessions during 1982-83. 

As a follow-up to these group dissemination sessions, 251 follow-up 
sessions were conducted for individual school staffs which requested additional 
help in replicating a program. 

The programs are also considered demonstration sites, and thousands of 
visitors have observed them in action. During 1982-83, for example, 1,158 
educators visited the demonstration programs. 

Although the State Department of Education expects each program to dis- 
seminate information in a variety of ways, the Department al&o organizes 
and coordinates dissemination programs throughout California. Between 1970 
and 1982, two major dissemination conferences were held each year, one in 
northern California and one in southern California. The demonstration programs 
present workshop sessions and distribute curriculum materials to about 1,000 
educators at these yearly session3. Beginning in 1976, two additional, smaller 
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dissemination sessions were held annually in the more remote regions of Califs 

Departmer^lr '"T? dissemination meting, have been coordinated by the 
Department in Alturas, Bakersfield, Bishop, Chico Crescent n i-« pi r + 

Jos r e eka San r L S r' "TTV ^T^ESt^'S ^go^'san 

Jm;,T£. Santa Barbara ' Santa clara ' santa Maria « ■"*•"•«. su-- 

$614 Tel tTmi-l^TJlil 1 ^ P^ ect / irector s "ported that they spent 
Cf IITLa \ I \ destination and replication activities. Their reports 

indicated that the average program spent 17 percent of its budget for this 
purpose. The range wa£ from 6 percent to 39 percent. But this figure does not 
induce any regular salary monies for project directors, teachers or aides 
Some project directors report that half of their time is spent on 'preprint 

llott ll'Jr lstributin g P rinted materials, and, in general, teaching others 
about their programs. For this reason, it is difficult to know exfctly Sow much 
of each demonstration program's budget should be considered spent for dissemina- 

ivicLrL con ?d n ere C ; iV Jn ieS ' ^ ^ ° f th ° S6 in disLmina ion 

the I JSl! [?266 listed ' 3CtU31 ^ W ° Uld be "^ierably higher than 

on the lllltl) "I COnducted ^ the State Department of Education in 1978 

on the dissemination and replication activities of the demonstration programs 

wh U h: 'ha^ 3 o^t^t SS 1° I S T le °l lndividuals in -demonstration TcZi. 
w..u nao nao contact with at least one demonstration program. First dlracrors 

^fn/L SUbmlt naMS ° £ " h ° th =y had ™«»n to believe 'mlgnt oe 

rtL?ved de rr r r t Ksrclre S c E r<> ° ' "S' !"»'-■ * Us < ° f "« 3 .°°° ' «• »« 

..000 ^m"Hlfu:rL%eSive e ^e";n^ti1„^ l r:. ClM * ^ ° f 

The questionnaires were distributed in March 1978, and 343 usable onesUnn 

Te u r rned Wer 63 retUrned f ° U ° Wing r6SUltS ' 0f tho8e £ tL^airS 

programs' in '1 "J?* they Were 3CtUally U8in * materials/ideas from 

represented s^tuatS 5" 37 PerCent ° f the COn * leted questionnaires 

\ ed or 8 L ^" ions /here demonstration program materials/ideas had not yet 
been used or had been tried but were not in use at the time the questionnaire 

respond £; t r e ll0W " UP t f leph ° ne C3llS Were made t0 50 indivil U a e L Sno L 
responded that they were using program materials and ideas. Nine of the schools 

father SiS **l ? ' info ™ ation the questionnaires and to 

A s™' i data regarding the replication of demonstration programs. 

A summary of selected findings from the study follows: 

Extent of replication 

who sMd^hr" 218 Per80ns ' or 63 P ercent °f the completed questionnaires, 
tJat Tr? " Sing c pr ° gram material/ideas themselves. It is probable 

limitaJions 3 USerS W3S consid «ably higher because of sample 
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The questionnaire respondents reported passing along materials and 
information to more than 2,500 persons. Questionnaire responses showed 
that over 35,000 students in replicating schools had been involved in these 
programs. This does not represent the total number for the state, of 
course, but reflects only the findings from the questionnaire sent to a 
small sample of schools. Project replications are located over the entire 
state — in small and large districts and in urban and rural areas. 

The characteristics of replicating schools were: 

No clear-cut pattern of demographics was found. 

A total of 87 percent of the schools that attempt replications are 
using some additional financial support from local, state, and/or 
federal sources for the programs. Just over half of the sample receiv- 
ing ESEA Title I or SB 90 EDY funds use some of the funds to support 
the replications. Almost a quarter of the responding schools have 
special district funds, and 66 percent of these schools use some of 
this money to replicate demonstration programs. 

In effective replication, schools typically reported: 

Support by the school principal. 

More than one contact with demonstration programs. 

Someone had been made responsible for the program at the school. 

The program has been modified to fit local school needs. 

In rating the program's effect on student achievement, 95 percent cf 
the responses noted improvement. Ninety-three percent or more of the 
responses were positive regarding improvement of students 1 attitudes. 

Most of the procedures used to disseminate information about demonstra- 
tion programs received favorable ratings, with visits to the programs and 
in-service presentations receiving the highest ratings. Few schools 
reported that they had chosen to replicate an entire program. Apparently, 
most schools attempted to modify programs to meet the special needs of 
their own situations and to encourage greater interest and pride among 
their own staffs. Although over half of the respondents claimed that their 
programs represented total replication or replication with only minor 
mo iifi cations, most schools seemed to be using a substantial amount of 
material from the demonstration programs without necessarily following the 
procedures from those programs. 

The three characteristics most often selected on the questionnaire 
which promoted and facilitated replications were: 

The material was practical. 

Ws could get copies of any parts of the program we needed. 
It was easy to understand what was being done. 

Respondents interviewed stressed the third point as most important. 
Interviewees also noted the following: 

Materials designed to fit specific, described needs and to meet 
specific, described purposes 

Er|c 17 23 



- Clearly delineated processes 

Readily available materials 
~ A helpful program director 

Records which document claims of effectiveness ratings 

It would appear that the in-service presentations offered an introduction 
to the programs, and they provided motivacion and an opportunity to examine 
sample materials. Once the selection and the commitment had been made, a visit 
to the demonstration program became extremely important— particularly where a 
total program (as opposed to sets of materials) was to be replicated. 
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SUMMARY OF FINDINGS 

The following summary appears warranted based on the foregoing findings 
and descriptions: 

The legislative intent of the demonstration programs in reading and 
mathematics is being carried out effectively. 

Students from disadvantaged areas (low-income, high mobility, low student 
achievement) show consistent achievement improvements in reading and 
mathematics according to CTBS scores in grades seven, eight, and nine. 

Demonstration programs have become more effective o/er time with respect 
to showing greater achievement gains and greater cost-effectiveness. 

Replication and dissemination activities are extensive and ongoing. 

RECOMMENDATIONS 

The findings clearly show that the demonstration programs in reading and 
mathematics have been effective in promoting student achievement. For this 
reason, they should be allowed to continue their exemplary, innovative practices 
and dissemination activities. Moreover, several additional options exist to 
improve the programs while maintaining their basic purposes. These specific 
options are: 

I. Obtain new information on the existing demonstration program practices 
by: 

Collecting, organizing, and documenting instructional practices and 
curricular processes for statewide dissemination and for integration 
into state curricular frameworks (estimated cost: $300,000 one year 
only). 

- Developing and conducting an evaluation aimed at assessing the extent 

to which demonstration program models can be replicated in nondemonstra- 
tion schools (estimated cost: $300,000 for & two-year study). 

- Planning and conducting an evaluation to determine the long-term 
effect of improved student achievement in reading and mathematics in 
the seventh, eighth, and ninth grades on high school academic 
performance. 

II. Expand demonstration program practices in reading and mathematics into new 
schools not represented previously in the demonstration programs (estimated 
cost: $140,000 annually per school). 

III. Develop demonstration programs in new curricular areas, for example, 

science, using the techniques and processes established by the programs 
for reading and mathematics projects (estimated cost: $140,000 annually 
per school). 
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APPENDIX I 
SB 1155 (Chapter 1270 of Statutes 1983) 



Ch. 1270 

PART 34. EVALUATION AND SUNSETTINC OF PROGRAMS 

SEC; 9. Section 62000 of the Education Codeis repealed. 
SEC. 10.^ Section 62000 is added to the Education Code, to read: 

62000. "Sunset" and "sunset date" as used in this part, mean the 
date on which specific categorical programs cease to be operative 
and the provisions of Sections 62002, 62003, 62004, 62005, and 62005.5 
govern program funding. 

The following educational programs shall cease to be operative on 
the date. specified, unless the Legislature enacts legislation to 
continue the program after the review prescribed in Section 62006: 

(a) Sunset date of June 30, 1985: 

(1) Demonstration programs in reading and mathematics. 

(2) Local staff development and teacher education and computer 
centers. 

(3) Educational technology. 

(4) Professional development centers. 

(5) Instructional materials. 

(b) Sunset date of June 30, 1986: 

(1) Miller-Unruh Basic Reading Act of 1965. 

(2) School improvement program. 

(3) Indian early childhood education. 

(4) Economic impact aid. 

(5) Bilingual education. 

(c) Sunset date of June 30, 1987: 

(1) Special education. 

(2) Gifted and talented education. 

(d) Sunset date of June 30, 1988: 

(1) Child development and preschool programs. 

(2) Adult education. 

(3) Indian education centers. 

SEC. 11. Section 62000.5 of the Education Code is repealed. 
SEC. 12. Section 62001 of the Education Code is amended to read: 

62001. (a) The intent of the Legislature in enacting this section 
is to assure a thorough review of programs listed in Section 62000 and 
the funding sources thereof, so that they most effectively, efficiently, 
and economically meet the needs of pupils and improve schools. It 
is not the intent of the Legislature in enacting this section to remove 
resources frcm students with special needs, or to cease efforts to 
improve the schools. 

(b) It was the intent of the Legislature in enacting this part to 
ensure that each categorical program is thoroughly reviewed. It was 
also intended that programs would be reviewed the year prior to the 
sunset date so that school districts and county offices of education 
would not be subject to unnecessary confusion and disruption in 
planning and administering those programs. 

(c) It is further the intent of the Legislature that the legislative 
review of programs specified in Section 62000 shall consider the 
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appropriateness of state administration by agencies other than by the 
State Department of Education, such as the Commission on Teacher 
Crcdcntialing and the Chancellor of the California Community 
Colleges. 

SEC. 12.5. Section 62002.5 of the Education Code is amended to 
read: 

62002.5. Parent advisory committees and school site councils 
which are in existence pursuant to statutes or regulations as of 
January 1, 1979, shall continue subsequent to the termination of 
funding for the programs sunsetted by this chapter. Any school 
receiving funds from Economic Impact Aid or Bilingual Education 
Aid subsequent to the sunsetting of these programs as provided in 
this chapter, shall establish a school site council in conformance with 
the requirements in Section 52012. The functions and responsibilities 
of such advisory committees and school site councils shall continue 
as prescribed by the appropriate law or regulation in effect as of 
January 1, 1979. 

SEC. 13. Section 62006 of the Education Code is amended to read: 
62006. (a) The Legislature shall begin immediately a detailed 

study which shall ensure that each funding source and program be 

scrutinized regarding, but not limited to, the: 

(1) Appropriateness of identification formulas in determining 
which children have special needs. 

(2) Appropriateness of' allocation formulas and adequacy of 
funding. 

(3) Effectiveness of programs. 

(4) Appropriateness of local control. 

(5) Appropriateness of state lev.el involvement in monitor, 
review, and auditing to assure that funds are being used efficiently, 
economically, and legally. 

(6) Appropriateness of costs of administration at all levels of 
operating these programs. 

(7) Appropriateness of State Department of Education 
administration of categorical programs. 

(8) Interrelationships between and among state and federal 
categorical programs, as appropriate. 

(9) Characteristics of the target population being served. 

(10) Need for the program. 

(11) Purpose and intent of the program. 

(b) In order to facilitate the legislative review, reports shall be 
developed and submitted to the Legislature pursuant to subdivisions 
(c), (e), and (0. The reports for programs scheduled to sunset in 
1985 shall be submitted to the Legislature by December 1, 1983; for 
programs scheduled to sunset in 1986, the reports shall be submitted 
by September 15, 1984; for programs scheduled to sunset in 1987, the 
reports shall be submitted by September 15, 1985; for programs 
scheduled to sunset in 1988, the reports shall be submitted by 
September 15, 1986. 
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The report by any agency in any given year may comment, within 
a single reco* c, on all programs scheduled to sunset in the applicable 
year. 

(c) The State Department of Education shall submit a report on 
the applicable programs pursuant to the schedule provided in 
subdivision (b) and shall also submit a copy of each report to the 
Legislative Analyst. The report shall contain, but not be limited to, 
the following: 

(1) A description of the programs, including narrah/e 
descriptions of how they are typically operated at the local level and 
how they are administered at the state level. 

(2) The history of the program or programs and previous 
legislative action. 

(3) Rele vant statistical data, including enrollment and fiscal data. 

(4) Related federal programs, and any provisions of federal law 
which may be appropriate for the Legislature to consider in its 
review of the state programs. 

(5) Whether there is an unmet need for the intended purposes of 
the program and, if any, the estimated cost of serving that unmet 
need. 

(6) Findings regarding the program, addressing as many of the 
issues identified in subdivision (a) as is possible. To the extent 
appropriate, as determined by the State Department of Education, 
the report .shall include comments on whether any identified 
problems are implementation issues, or issues that warrant revision 
of law or regulations. 

(7) Recommendations of ways to improve the proj^ am while 
maintaining its basic purposes. 

(d) The Legislative Analyst shall review the report submitted by 
the State Department of Education and, no later than 90 days 
following the receipt of each report, shall submit findings, 
comments, and recommendations, as the Legislative Analyst 
determines appropriate, regarding the program, addressing as many 
of the issues identified in subdivision (a) as the Legislative Analyst 
determines is possible. To the extent determined appropriate by the 
Legislative Analyst, the report shall include comments on whether 
any identified problems are implementation issues or issues that 
warrant revision of the law or regulations and shall include 
recommendations of ways to improve the programs while 
maintaining its basic purposes. 

(e) The Legislative Counsel shall submit a report on the 
applicable programs, pursuant to the schedule provided in 
subdivision (b). The report shall include, but not be limited to, the 
following: 

(1) A summary of the law regarding the programs, including 
applicable regulations. 

(2) A summary of related federal law and regulations, if any. 

(3) A summary of related court decisions, if any. 
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(4) A summary of any federal provisions or court decisions which 
piace constraints on the Legislature's alternatives. 

(f) Each temporary advisory committee established pursuant to 
Section 62006.5 shall submit a report on the applicable programs, 
pursuant to the schedule provided in subdivision (b). The report 
shall make findings and recommendations on as many of the issues 
identified in subdivision (a), as is possible. 

SEC. 14. Section 62006.5 is added to the Education Code, to read: 

62006.5. (a) There aie hereby established the following four 
temporary advisory committees, each to serve for the specified 
period of time: 

(1) A temporary advisory committee on the programs scheduled 
to sunset on June 30, 1985. This advisory committee shall serve for the 
period September 1, 1983 through December 31, 1983. 

(2) A temporary advisory committee on the programs scheduled 
to sunset on June 30, 1986. This advisory committee shall serve for the 
period January 1, 1984 through December 31, 1984. 

(3) A temporary advisory committee on the programs scheduled 
to sunset on June 30, 1987. This advisory committee shall serve for the 
period January 1, 1985 through December 31, 1985. 

(4) A temporary advisory committee on the programs scheduled 
to sunset on June 30, 1988. This advisory committee shall serve for the 
period January 1, 1986 through December 31, 1986. 

(b) Kach temporary advisory committee shall consist of 12 
members selected as follows: 

(1) Three members selected by the Speaker of the Assembly who 
shall ensure that his or her appointees consist of one administrator 
or school board member, one nonadministrative school employee, 
and one parent, student, or community member. 

(2) Three members selected by President pro Tempore of the 
Senate who shall ensure that his or her appointees consist of one 
administrator or school board member, one nonadministrative 
school employee, and one parent, student, or community member. 

(3) Six members selected by the Governor as follows: 

(i) Two administrators or school board members. 

(ii) Two nonadministrative school employees. 

(iii) Any combination of two parents, students, or community 
members. 

(c) Members of each advisory committee shall serve without 
compensation, except for reimbursement of expenses. 

(d) The State Department of Education shall provide staff 
support to each advisory committee. 

(e) This section shall remain in effect only until January 1, 1987, 
and as of such date is repealed, unless a later enacted statute, which 
is chaptered before January 1, 1987, deletes or extends such date. 

SEC. 15. Part 36 (commencing with Section 64000) is added to 
the Education Code, to read: 
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APPENDIX II 

DEMONSTRATION PROGRAMS IN READING AND MATHEMATICS 
1983-84 



District 


| irir | 


School 


i Address I 


project Director 


1 Telephone 


Bflkfirnf i&ld 


1 1 

IR j 


Coimton Ttininr Hioh 


I 3211 Pico St., 8akersfield, CA 93306 I 


Bill Mr»L pan 


1(805) 


872-4690 


lj a f\ lj ^ o i xuxu 


IR 1 


Sierra Junior Hioh 


I 3017 Center St., Bakersfield, CA 93306 I 


Barbara Clark 


1(805) 


323-4838 


Barkalnv* 


IR 1 


Willard Junior Hiah 


I 2425 Stuart St., Berkeley, CA 94705 I 


Dianna Panne v 

\* ■* Ul II 1 U 1 ul II Iw T 


1(415) 


644-6330 


Colton 


IRMI 


Terrace Hills Jr. Hiah 

Iwlia^u 1 1 Alio • 1 IA Mi 1 


I 22579 DeBerry St., Colton, CA 92324 I 


Marilvnn Parmn 


1(714) 


824-4245 


f-'l Ranchn* 


IR 1 


North Park Middle 
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Earl Watson, Manager, Demonstration Programs 

Office of Special Curriculum Services 

California State Department of Education 

721 Capitol Mall 

Sacramento, CA 95814 

(916) 323-0404 gj^ 
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